Assurance of learning : the role of work integrated learning and industry partners

In the partnering with students and industry it is important for universities to recognize and value the nature of knowledge and learning that emanates from work integrated learning experiences is different to formal university based learning. Learning is not a by-product of work rather learning is fundamental to engaging in work practice. Work integrated learning experiences provide unique opportunities for students to integrate theory and practice through the solving of real world problems. This paper reports findings to date of a project that sought to identify key issues and practices faced by academics, industry partners and students engaged in the provision and experience of work integrated learning within an undergraduate creative industries program at a major metropolitan university. In this paper, those findings are focused on some of the particular qualities and issues related to the assessment of learning at and through the work integrated experience. The findings suggest that the assessment strategies needed to better value the knowledges and practices of the Creative Industries. The paper also makes recommendations about how industry partners might best contribute to the assessment of students’ developing capabilities and to continuous reflection on courses and the assurance of learning agenda.

Hunters & gatherers : strategies for curriculum mapping and data collection for assurance of learning

Assurance of learning is a predominant feature in both quality enhancement and assurance in higher education. Assurance of learning is a process that articulates explicit program outcomes and standards, and systematically gathers evidence to determine the extent to which performance matches expectations. Benefits accrue to the institution through the systematic assessment of whole of program goals. Data may be used for continuous improvement, program development, and to inform external accreditation and evaluation bodies. Recent developments, including the introduction of the Tertiary Education and Quality Standards Agency (TEQSA) will require universities to review the methods they use to assure learning outcomes. This project investigates two critical elements of assurance of learning: 1. the mapping of graduate attributes throughout a program; and 2. the collection of assurance of learning data. An audit was conducted with 25 of the 39 Business Schools in Australian universities to identify current methods of mapping graduate attributes and for collecting assurance of learning data across degree programs, as well as a review of the key challenges faced in these areas. Our findings indicate that external drivers like professional body accreditation (for example: Association to Advance Collegiate Schools of Business (AACSB)) and TEQSA are important motivators for assuring learning, and those who were undertaking AACSB accreditation had more robust assurance of learning systems in place. It was reassuring to see that the majority of institutions (96%) had adopted an embedding approach to assuring learning rather than opting for independent standardised testing. The main challenges that were evident were the development of sustainable processes that were not considered a burden to academic staff, and obtainment of academic buy in to the benefits of assuring learning per se rather than assurance of learning being seen as a tick box exercise. This cultural change is the real challenge in assurance of learning practice.

Communities of practice : building organisational capability through an undergraduate assurance of learning program

This study explores academic perceptions of organizational capability and culture following a project to develop a quality assurance of learning program in a business school. In the project a community of practice structure was established to include academics in the development of an embedded, direct assurance of learning program affecting more than 5000 undergraduate students and 250 academics from nine different disciplines across four discipline based departments. The primary outcome from the newly developed and implemented assurance of learning program was the five year accreditation of the business school’s programs by two international accrediting bodies, EQUIS and AACSB. This study explores a different outcome, namely perceptions of organizational culture and individual capabilities as academics worked together in teaching teams and communities. This study uses a survey and interviews with academics involved, through a retrospective panel design consisting of an experimental group and a control group. Results offer insights into communities of practice as a means of encouraging new individual and organizational capability and strategic culture adaptation.

Australian higher education evaluation through assurance of learning

A collaborative research project conducted by five Australian universities inquired into the philosophy and motivation for Assurance of Learning (AoL) as a process of education evaluation. Associate Deans Teaching and Learning representing Business schools from twenty-five universities across Australia participated in telephone interviews. Data was analysed using NVIVO9. Results indicated that articulated rationale for AoL was both ensuring that students had acquired the attributes and skills the universities claimed they had, and the philosophy of continuous improvement. AoL was motivated both by ritualistic objectives to satisfy accreditation requirements and virtuous agendas for quality improvement. Closing-the-loop was emphasised, but was mostly wishful thinking for next steps beyond data collection and reporting. AoL was conceptualised as one element within the larger context of quality review, but there was no evidence of comprehensive frameworks or strategic plans.

Assurance of learning – are academics and senior management singing from the same hymn sheet?

Assurance of learning (AoL) is an important process in quality education, designed to measure the accomplishment of educational aims at the core of an institution’s programs, whilst encouraging faculty to continuously develop and improve the programs and courses. This paper reports on a study of Australian business schools to investigate current AoL practices through semi structured interviews with senior faculty leaders followed by focus group interviews with groups of senior program leaders and groups of academic teaching staff. Initial findings indicate there are significant challenges in encouraging academic staff to commit to the process and recognise the benefits of assuring learning. The differences in understanding between the various leaders and the academics were highlighted through the different focus groups. Leaders’ stressed strategic issues such as staff engagement and change, while academics focussed on process issues such as teaching graduate attributes and external accreditation. Understanding the differences in the perspectives of leaders and faculty is important, as without a shared understanding between the two groups, there is likely to be limited engagement, which creates difficulties in developing effective assurance of learning processes. Findings indicate that successful strategies developed to foster shared values on assurance of learning include: strong senior leaders’ commitment; developing champions among program and unit level staff; providing professional development opportunities; promoting and celebrating success and effectiveness; and ensuring an inclusive process with academics of all levels collaborating in the development and implementation of the process.

Building sustainable and effective assurance of learning process in a changing higher education environment

In a study of assuring learning in Australian Business Schools, 25 Teaching and Learning Associate Deans were interviewed to identify current issues in developing and measuring the quality of teaching and learning outcomes. Results indicate that for most institutions developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program required knowledge creation and the building of new inclusive processes. Common elements of effective practice, namely those which offered consistently superior outcomes, included: inclusive processes; embedded graduate attributes throughout a program; alongside consistent and appropriate assessment. Results indicate that assurance of learning processes are proliferating nationally while quality of teaching and learning outcomes and in the processes for assuring it is increasing as a result.

A preliminary investigation into technology and processes facilitating the assurance of learning

This paper reports on the outcomes from a preliminary evaluation of technologies and processes intended to support the Assurance of Learning initiative in the business faculty of an Australian university. The study investigated how existing institutional information systems and operational processes could be used to support direct measures of student learning and the attainment of intended learning goals. The levels at which learning outcomes had been attained were extracted from the University Learning Management System (LMS), based on rubric data for three assessments in two units. Spreadsheets were used to link rubric criteria to the learning goals associated with the assessments as identified in a previous curriculum mapping exercise, and to aggregate the outcomes. Recommendations arising from this preliminary study are made to inform a more comprehensive pilot based on this approach, and manage the quality of student learning experiences in the context of existing processes and reporting structures.

Designing an e-portfolio for assurance of learning focusing on adoptability and learning analytics

The Assurance of Learning for Graduate Employability framework is a quality assurance model for curriculum enhancement for graduate employability, enabling graduates to achieve "the skills, understandings and personal attributes that make [them] more likely to secure employment and be successful in their chosen occupations to the benefit of themselves, the workforce, the community and the economy" (Yorke, 2006). Of particular note is the framework's dependence on three foundations, including easy access to integrated and accessible tools for staff and student self-management. In other words, this approach to curriculum quality depends on staff and student access to tools that enable them to self-manage their learning. This paper examines two aspects which informed the design of a student e-portfolio system, iPortfolio, intended for students' self-management of their learning, particularly recording evidence of their achievement of capabilities. The paper focuses on two particular considerations in the design of the iPortfolio: adoptability and learning analytics. Adoptability means the phase preceding adoption, whether students have the devices, platforms and technology skills to be able to use such an innovation. The iPortfolio also facilitates learning analytics: it has the capability to gather data related to learning indicators for course quality assurance purposes. Both adoptability and analytics are very dynamic fields: new devices, platforms and applications constantly spark changes in user habits, and policy changes mean institutions need to be able to provide new data, often at short notice. In the conclusion, the paper suggests how tools such as the iPortfolio can be designed for 'future proofing' and sustainability.

Impact of Assurance of Learning (AOL) in programming course for novices

One of the aims of any higher education institution is to align its curriculum with program learning goals. Programs which ensure proper learning have positive effects on students, instructors, departments and also on the higher education institution itself. This paper discusses the implementation and effects of Assurance Of Learning (AOL) processes on introductory programming (IP) courses. It elaborates five stages of AOL to align program learning goals with IP curriculum. Then, it discusses how the AOL process identifies shortcomings in the assessment methods of IP courses. Furthermore, it enlightens how the assessment findings, as a result of the AOL process, provide mechanisms to address the drawbacks during the delivery of such courses. Feedback on

The Effectiveness of Learning Portfolios: A Study of Quality Assurance Programs of Selected Health Regulatory Colleges in Ontario

Health regulatory colleges promote quality practice and continued competence through Quality Assurance (QA) programs. For many colleges, a QA program includes the use of portfolios that incorporate self-directed learning. The purpose of this study was to determine some of the issues surrounding the effectiveness of QA portfolio programs. The literature review revealed that portfolios are valuable tools, but gaps in knowledge include a comparative analysis of QA programs and the perspective of regulatory college administrators. Data were collected through interviews with 6 administrators and a review of 14 portfolio models described on college websites. The results from the two data sources were applied to Robert Stake's responsive evaluation framework to identify issues related to the portfolio's effectiveness (Stake, 1967). The learning components of portfolios were analyzed through the humanist and constructivist lenses. All 14 portfolio models were found to have 3 main components: self-diagnosis, learning plan and activities, and self-evaluation. However, differences were uncovered in learners' autonomy in selecting learning activities, methods of portfolio evaluation, and the relationship between the portfolio and other QA components. The results revealed a dual philosophy of learning in portfolio models and an apparent contradiction between the needs of the individual learner and the organization. Paths for future research include the tenuous relationship between competence and learning, and the impact of technical approaches on selfdirected learning initiatives. A key recommendation is to acknowledge the unique identity of each profession so that health regulatory colleges can address legislative demands and learner needs.

Student self-assessment of performance : a useful learning tool for students and instructors?

Assurance of Learning is a focus for many Business Schools to meet AACSB requirements. Prior research has examined student self-assessment as a measure of learning in this context, but we propose instead that this is a useful ‘learning tool’ for students and instructors that can help improve student performance and curricula. We examined self-assessment in a third year undergraduate e-marketing course and found a positive, though weak relationship between student self-assessment and instructor ratings on intended learning goals and criteria for a written assignment. Students who self-assessed tended to perform better in the assessment item and course than those who did not self-assess. But less competent students were more inaccurate in their self-assessment, suggesting a need to enhance understanding of marking criteria and performance standards within this group. Implications for educators are discussed.

The practice of quality assuring learning in higher education

There remains a lack of published empirical data on the substantive outcomes of higher learning and the establishment of quality processes for determining them. Studies that do exist are nationally focused with available rankings of institutions reflecting neither the quality of teaching and learning nor the diversity of institutions. This paper describes two studies in which Associate Deans from Australian higher education institutions and focus groups of management and academics identify current issues and practices in the design, development and implementation of processes for assuring the quality of learning and teaching. Results indicate that developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program necessitates knowledge creation and new inclusive processes. Common elements supporting consistently superior outcomes included: inclusivity; embedded graduate attributes; consistent and appropriate assessment; digital collection mechanisms; and systematic analysis of outcomes used in program review. Quality measures for assuring learning are proliferating nationally and changing the processes, systems and culture of higher education as a result.

From policy borrowing to implementation : an illuminative evaluation of learning and teaching in higher education in Australia (2002 to 2008)

This study documents and theorises the consequences of the 2003 Australian Government Reform Package focussed on learning and teaching in Higher Education during the period 2002 to 2008. This is achieved through the perspective of program evaluation and the methodology of illuminative evaluation. The findings suggest that the three national initiatives of that time, Learning and Teaching Performance Fund (LTPF), Australian Learning and Teaching Council (ALTC), and Australian Universities Quality Agency (AUQA), were successful in repositioning learning and teaching as a core activity in universities. However, there were unintended consequences brought about by international policy borrowing, when the short-lived nature of LTPF suggests a legacy of quality compliance rather than one of quality enrichment.

A stakeholder analysis of one institution’s assuring learning experience – is the promise and the practice worth the price?

Universities around the world are rushing to implement assurance of learning policies and practices with varying degrees of success. One School investigated its own policy and practice development through the eyes of its key stakeholders to identify whether the practice was worth the price. Findings indicate that although the key stakeholders considered different needs and viewed their experiences differently, value did abound and was in the eye of the beholder.